Analyzing the understandability of Requirements Engineering languages for CSCW systems: A family of experiments

Context: A collaborative system is a special kind of software whose users can perform collaboration, communication and collaboration tasks. These systems usually have a high number of non-functional requirements, resulting from the users’ need of being aware of other users with whom to collaborate, that is, the workspace awareness. Objective: This paper aims at evaluating two Requirement Engineering languages i* and CSRML (an extension of i*) in order to determine which is the most suitable one to specify requirements of collaborative systems, taking into account their special characteristics regarding collaboration and awareness. Method: We performed a family of experiments comprising an original experiment and two replicas. They were performed by 30, 45 and 9 Computer Science students, respectively, from Spain and Argentina. These subjects filled in two understandability questionnaires once they analyzed the requirements models of two systems: an e-learning collaborative system and a conference review system with some collaborative aspects support. Both models were specified by using the evaluated languages. Results: The statistical analysis of the family of experiments showed that the understandability was higher for the models specified with CSRML than for those specified with i*, especially for collaborative aspects. This result was also confirmed by the meta-analysis conducted. Conclusions: CSRML surpasses i* when modeling collaborative systems requirements models due to the specific expressiveness introduced to represent collaboration between users and awareness and the new resorts to manage actors and roles. Highlights. > Two Requirements Engineering languages, namely i* and CSRML, were analyzed attending to understandability criteria of requirements for CSCW systems. > A family of experiments at three different universities was conducted. > The participants tried to understand the requirements models of two different systems. > The obtained results were summarized by means of meta-analysis techniques. > Results show that CSRML is more understandable than i* for dealing with requirements of CSCW systems.